Hyperspectral screen-image-synthesis meter with scattering-noise suppression

The screen image synthesis (SIS) meter was originally proposed as a high-speed measurement tool, which fused the measured data from multiple sample-rotational angles to produce a whole-field measurement result. However, it suffered from stray light noise and lacked the capability of spectrum measurement. In this study, we propose an SIS system embedded with a snapshot hyperspectral technology, which was based on a dispersion image of the sparse sampling screen (SSS). When a photo was captured, it was transformed and calibrated to hyperspectral data at a specific sample-rotational angle. After the hyperspectral data in all sample-rotational angles were captured, an SIS image-fusion process was then applied to get the whole field hyperspectral data. By applying SSS to the SIS meter, we not only create a screen image synthesis hyperspectral meter but also effectively address the issue of stray-light noise. In the experiment, we analyze its correctness by comparing the hyperspectral value with a one-dimensional spectrum goniometer (ODSG). We also show the 2D color temperature coefficient distribution and compare it with the ODSG. Experimental results also demonstrate the feasibility in terms of both spectrum distribution meter and color coefficient temperature distribution meter.


SIS hyperspectral system
Figure 1 depicts the SISH system is composed by a snap-shot hyperspectral system and a sample-rotational system.The snap-shot hyperspectral system is composed by a camera, a grating, a band pass filter (BPF) and a sparse-sampling screen (SSS).The irradiance captured by the CMOS sensor corresponded to the radiance from each position of the screen and was captured by the camera.The sample rotational system is composed by Motor1, Mortor 2, a rail system and a sample mount.A Cartesian coordinate system (x 0 , y 0 , z 0 ) is defined in the snap-shot hyperspectral system.The sample-rotational system is driven using two motors.Motor 1 rotates all components of the sample-rotational system along the red axis.A rail system is fixed on the loading side of Motor 1 and is driven by Motor 2, which rotates the load components along the blue axis.(x 1 , y 1 , z 1 ) is the cartesian coordinate system defined on the loading side of Motor 1.When Motor 1 rotates, all the components loaded by Motor 1 rotate ω 1 along the y 0 -axis.(x 2 , y 2 , z 2 ) is the cartesian coordinate system defined on the loading side of the rail system.When Motor 2 rotates, the sample rotates ω 2 along the x 1 -axis.Motor 1 rotates from −66 • to 66 • with a step of 33 • .Motor 2 rotates from -150 • to 180 • with a step of 30 • .The two motors rotate to 60 angles to cover the 4π solid angle surrounding the sample.After the progress of parameter optimization and system calibration, the image reconstruction algorithm is used to obtain the 2-D light distribution.

Snap-shot hyperspectral technique
Figure 2a shows the schematic diagram of the snap-shot hyperspectral system.The camera consists of a CMOS image sensor and a lens, and it forms the image of the SSS on the CMOS image sensor.A grating mounted in front of the lens is used to disperse the image.The SSS is used to depress the SRSL noise and to perform the spectrum acquiring by Spatial multiplexing.The SSS is composed by a multi hole black stick (MHBS), a white stick (WS), and an AR coating glass (ARG) with size 601 (H) ×496 (W) ×3 (D) (mm 3 ).The optical property of the WS is scattering with Lambertian distribution.The band pass filter (BPF) in front of the grating limits the detection band within 390-710 nm, to remove the noise caused by field curvature aberration.The material of both the MHBS and WS are polyvinyl chloride (PVC).The distance between each hole is 10mm, and the hole diameter is 0.8mm.It makes the scattering reflection of the screen decreasing to 0.6% .The SRSL noise is depressed Figure 1.The architecture of the SIS hyper-spectrum system is composed with a sample-rotational system and a snap-shot hyperspectral system.It captures the spectrum distribution with 4π solid angle.accordingly.Figure 2b shows a design of the square holes array.The total number of holes in the MHBS is 2, 401.However, series distortion aberration of the off-axis dispersion image makes the spectrum image banding.The neighboring first-order spectrums overlap with each other, as shown in Fig. 2c.The spectrum detection of these off-axis holes thus failed.To improve the overlapping problem, we calculate the gradient of all the first order spectrum and increase the distance between the holes along the orthogonal direction of the gradient.Figure 2d shows the new design.Because of the distance increase, the total area of the holes is expanded.Some of the holes in the new design are located outside the SSS and are removed.So, the total number of the holes in the new design decreases to 2233. Figure 2e shows the dispersion image of the new design, and the off-axis spectrum overlapping problem are resolved.
For the imaging system shown in Fig. 2a, the object distance is d 2 , and the image distance is d 3 (as shown in Eq. (1).The light beam dispersed by the grating with the dispersion angle �θ( ) .The corresponding shifting of the dispersed point is In the linear space invariant region of the image system, the spectrum can be treated as a convolution of the image of the hole and the spectrum curve.Thus, the spectral resolvance (V) is limited by the hole size, and is written as The designed spectral resolution (R) is calculated from the spectral resolvance It shows smaller hole size and longer object distance ( d 1 ) lead to higher spectral resolvance.In this paper, the CMOS image sensor has 5 million pixels and the pixel pitch are 3.45 µ m; the lens has focal length 5 mm; d 2 = 750 mm; d 3 = 5.03 mm.The image of the hole occupies 3 × 3 pixels in the CMOS Image Sensor.The designed spectral resolution is 5 nm.The distance d 1 between the sample and the SSS is 500 mm, and the longest distance between the sample and the diagonal edge of the SSS is 612 mm.Therefore, the angular resolution is 0.9

Spectrum retrieval process
SSS(x 0 , y 0 ; ) is the hyperspectral data on SSS, and it is retrieved from a 2D photo G(ξ , η) , where (x 0 , y 0 ) is the position of the sampling hole on SSS, and (ξ , η) is the coordinate of the CMOS image sensor.The point spread function of the optical system comparing to the size of the sampling hole is small enough, and the location (ξ , η) of the dispersed image for the sampling hole located at (x 0 , y 0 ) is expressed as a quadratic function of .The relation between the 2D photo and the hyperspectral data can be expressed as We use a laser beam with wavelengths of 457nm, 532nm, and 671nm to pass through a diffuser sequentially.
The scattering light illuminated the SSS.And we capture the dispersed image of SSS to get G r (ξ , η) , G g (ξ , η) , and G b (ξ , η) .Because the three dispersed images are discrete points array.It is easy to measure the position (ξ , η) cor- responding to each sampling hole located at (x 0 , y 0 ) .Use the three photos, we have 6 equations to solve the coef- ficients, i.e., a(x 0 , y 0 ) ∼ f (x 0 , y 0 ) .Then, we can transform a captured photo G(ξ , η) to the hyperspectral raw data In the calibration process, we used a white light LED as the standard light source.We take a picture and calculate hyperspectral raw data SSS w (x 0 , y 0 ; ) using Eq. ( 5).Then we shifted a spectrometer (ISUZU OPTICS, ISM-Lux, corrected by ITRI-CMS) two-dimensionally to measure the spectrum distribution, i.e., SSS t (x 0 , y 0 ; ) .Thus, the calibration function was Therefore, when the hyperspectral meter was applied to measure an arbitrary light source, the calibrated hyperspectral was:

Image reconstruction algorithm of the SISH system
Because the imaging system is static, the sample should be rotated in steps to fill all detection angles.The rotational angles of Motor 1(ω 1 ) and Motor 2 ( ω 2 ) are expressed as functions of integer indices i and j respectively, When the sample rotates, we need to map detected hyperspectral distribution on the screen in Cartesian coordinate, i.e.SSS(x 0 , y 0 ; ) , to the captured hyperspectral data with sample-rotational angle in spherical coordinate, i.e.SSS i,j (θ, φ; ) .The Cartesian coordinates attached to the screen, Motor 1, and Motor 2 (as well as samples) are expressed as the span of the bases B 0 , B 0 , and B 1 , i.e., B 0 x 0 y 0 z 0 T , B 0 x 1 y 1 z 1 T , and respectively.The origin of the coordinate frames is located at the rotational center of the sample.For the rotation of Motors 1, and 2, the rotational matrix is expressed as follows: and where the subscripts B0 and B1 indicate that they are used as the basis of the corresponding rotational matrices.Using Eqs. ( 10) and ( 11), the intensity distribution captured with the Cartesian coordinates attached to the screen is transferred to the Cartesian coordinates attached to the sample 35 The Cartesian coordinates are mapped to spherical coordinates (Fig. 3) as follows: After the images for different rotational angles are captured at specific ω 1 and ω 2 , Eqs. ( 4)-( 7) is used to calculate the hypectral SSS(x 0 , y 0 ; ) .Subsequently, Eqs. ( 8)-( 14) are used to calculate the hyperspectral SSS i,j (θ, φ; ) .
Where, θ and φ is the zenith angle and the azimuth angle of the spherical coordinate, respectively.In case of a forward illumination sample, 30 pictures with various values of ω 1 and ω 2 are used to reconstruct the whole field hyperspectral data.This is expressed as follows: (5) SSS 0 (x 0 , y 0 ; ) = G(ξ(x 0 , y 0 ; ), η(x 0 , y 0 ; )).
(7) SSS(x 0 , y 0 ; ) = SSS 0 (x 0 , y 0 ; ) • C(x 0 , y 0 ; ).where W(θ, φ) is the number of repetitions of the superposition process.Equation ( 15) means SSS i,j (θ, φ; ) in all rotational angles are summed, and is divided by W(θ, φ) .The calculation of W(θ, φ) uses the same simulation codes as SSS i,j (θ, φ; ) .We replaced SSS(x 0 , y 0 ; ) with an all-ones matrix, where all entry is equal to one.The all-ones matrix is mapped to the spherical coordinate using Eqs.( 8)-( 12) to produce OCP i,j (θ, φ) , which is the area occupied by SSS i,j (θ, φ; ) .Then we sum OCP i,j (θ, φ) in each shot to calculate the number of repetitions Table 1 shows the descriptions of functions used for the hyperspectral retrieval and the SIS fusion.The hyperspectral retrieval process uses the captured photo G(ξ , η) to calculate SSS(x 0 , y 0 ; ) , which is the hyperspectral data on the SSS screen.The SIS fusion process maps SSS(x 0 , y 0 ; ) to spherical coordinate with considering the rotational angle of the sample ω 1 (i) and ω 2 (j) , and gets SSS i,j (θ, φ; ) .Then all SSS i,j (θ, φ; ) are fused to get I SIS (θ, φ; ).

System performance estimation methods
The distribution of coefficient of color temperature (CCT) is the key to photometry measurements, and it is convenient to show the whole measurement result using 2D Azimuth equidistant projection.So, we not only estimate the system performance of hyperspectral measurement, but also use the CCT distribution to estimate the system performance.The normalized correlation coefficient (NCC) is used to quantify the system performance of the hyperspectral measurement 66 .The NCC is defined as where I SIS and I ODG are the spectrum measured by the SISH and the one-dimensional spectrum goniometer (ODSG), respectively.ĪSIS and ĪODG are the mean values of the I SIS and the I ODG , respectively.m is the spectrum band number, and the value of m is 55 in this paper.For the performance of the color distribution measurement, we need to calculate the CCT distribution and the color coordinate.According to McCamy formula 67 , where c x , and c y are corresponding color coordinate in CIE 1931 x-y chromaticity coordinate system 68 .And then calculate the CCT using the cubic polynomial approximation 69 Finally, we use the average of the different (AD) and the different standard deviation (DSD) to evaluate the CCT measurement performance.Different from the unitless NCC, which is used to evaluate the similarity of two curves.The AD and the DSD in unit of color temperature (k) bring the sense of color accuracy and color precision.The AD is defined as Where N is the total amount of the measurement of θ T .F SIS is the CCT distribution or the color coordinate distribution based on the SISH and F ODG is the CCT distribution or the color coordinate distribution based on the ODSG.OCP i,j (θ, φ), CCT = −449n 3 + 3525n 2 − 6823.3n+ 5520.33, Table 1.The function list. .

Function Description
Hyperspectral retrieval G(ξ , η) Captured photo SSS W (x 0 , y 0 ; ) Hyperspectral raw data of a standard light source SSS t (x 0 , y 0 ; ) Hyperspectral raw data of a standard light source measured by a standard spectrometer C(x 0 , y 0 ; ) Clibration function SSS(x 0 , y 0 ; ) Hyperspectral data calculated from G(ξ , η) in Cartesian coordinate SIS fusion The rotational angle of motor1 and motor2 The rotational matrix of motor1 and motor2 SSS i,j (θ , φ; ) Hyperspectral data calculated from G(ξ , η) in spherical coordinate OCP i,j (θ , φ) The area occupied by SSS i,j (θ , φ; ) The number of repetitions

Experiment and performance evaluation
In the experimental process, we capture the spectrum distribution of different samples by using the SIS with aid of SSS, as shown in Fig. 1.In each rotational angle (ω 1 , ω 2 ) , a 2D photo G(ξ , η) was taken.Then, a calibrated hyperspectral SSS(x 0 , y 0 ; ) was calculated by putting G(ξ , η) into Eqs.( 4)- (7).It was then mapped to the spheri- cal coordinate and was expressed as SSS ( i, j)(θ, φ; ) by using Eqs.( 8)- (12).We count the number of repetitions W(θ, φ) in each position, and then apply these variables to Eq. ( 15) to calculate I SIS (θ, φ; ) .The calculation software is MATLAB.For sake of clear expression, we draw I SIS (θ, φ; ) on a disc using the azimuth equidistant projection, as shown in Fig. 3.The direction of (θ, φ) = (0, 0) is normal to the samples.In the experiment, since the measuring angular resolution is 0.9 • ∼ 1.2 • .We interpolate the measurement result to the spherical coordi- nate of (θ, φ) grid with the angular resolution 1 • .The calculation results produce a total of 180 × 180 spectrum curves.Figure 3b-d  In Fig. 3b-d, the sample is a Lab-packaged LED using hemisphere package.The color distribution of LEDs in "Hemisphere" package should be radially symmetrical.However, we deliberately make a "Bias Hemisphere" package to change the color distribution.As a result, the spectrum distributions for the three figures are obviously different.Figure 4 shows the per-three-bands image of the "Bias Hemisphere" package.The blue band (435-460nm) is obviously affected by the bias package, but other bands appear to be unaffected.Because the blue light is emitted by a small die and passes through the phosphor, the smaller etendue makes it easier to be shaped by the hemispherical package lens.Other wavelength bands are re-emitted by the phosphor.The large etendue makes it hard to be shaped by the hemispherical package lens.
In order to evaluate the system performance, we used a laser beam with wavelength 532nm to illuminate a diffuser and to capture the image G(ξ , η) .A smoothing process was applied to average each pixel with 8 surround- ing pixels, to get a new value of the pixel.The image after smoothing process is shown in Fig. 5a.It shows the hole distribution is sparse enough to separate the spectrum of each hole and makes sure the angular resolution follows the design value, i.e., 0.9 • to 1.2 • .However, the continuous temporal spectrum makes the image of the hole of different wavelength overlapping each other, the spectral resolution should be evaluated by the full width of half maximum (FWHM) of the point spread function (PSF) of the wavelength.The spectrum distribution corresponding to a hole location (x 0 , y 0 ) is calculated by Eq. ( 5).Thus, the path of a spectrum distribution in the photo (ξ(x 0 , y 0 ; ), η(x 0 , y 0 ; ) is illustrated with a yellow dash line in the figure.The PSF of the wavelength can be measured along the spectrum path, such as the blue curve in Fig. 5b.We apply Gaussian fitting with 95% fitting confidence, and get the orange curve.The full width of half maximum (FWHM) of the fitted Gaussian function is calculated accordingly.Figure 5c shows the statistic FWHM over total 2233 points.Because the fieldcurvature aberration is induced by the camera lens, the on-focusing depth changes with (x0, y0).It makes the FWHM varies from 5.6 to 12 nm, and the average FWHM is 10.1 nm.It turns out as the spectral resolution of the experimental results, and is about twice the theoretical spectral resolution calculated by Eq. ( 3).
We used the spectrum meter (ISUZU OPTICS, ISM-Lux, corrected by ITRI-CMS) as a standard device to build up an ODSG.It rotates around the sample to measure the comparison spectrum I ODG ( ) of the sample per 2 degrees, as shown in Fig. 6.
We tested four samples with SISH system and the ODSG.The four samples are Lab-packaged LEDs in the form of "Bias Hemisphere", "Hemisphere extended height coating (Hemisphere EHC)", "Conformal without lens" and "Cup".Where, "cup" and "Conformal without lens" were the conventional techniques to achieve uniform CCT distribution 67 .The "Hemisphere EHC" was then proposed to achieve better uniformity of CCT distribution 70 .However, to show the value of SISH meter, which can be applied to check the quality of design or manufacture,  we deliberately change the package parameter.For the Lab-packaged "Bias Hemisphere", we made the lens biasing from the center, to simulate a poor package quality.For the Lab-packaged "Hemisphere EHC", we deliberately choose the extended height of 1mm to produce poor CCT uniformity.
In the experiment, I ODG ( ) for φ = 0 • were taken by the ODSG, and I SIS ( i ) for φ = 0 • were taken by the SISH meter.Then we calculate the NCC values between the measured results of the I ODG ( ) and the I SIS ( i ) . Figure 7 shows for all types of packages, in the detection region |θ T | < 80 • , NCC values are higher than 95% .The system performance gets worse in large θ T because the intensity in large theta T are weak.It induces noise and degrades the NCC Especially, the package of "Bias Hemisphere" in the region θ T > 80 • , weak signal and dramatic spectrum variation lead to the drop of NCC.We also need to note int the package of conformal without lens, the signals with θ T < 80 • are so weak, such that the ODSG detects only the noise.It makes the comparison values I ODG ( i ) within these regions not trustable.So, we removed all the system evaluation within this region.
Because a single value CCT is calculated from the 1D spectrum, the 3D hyperspectral data can be transformed to the 2D CCT distribution, and thus be shown in a single 2D picture.And it is sensitive to spectrum variation.Besides, color distribution is an important parameter for the lighting and display manufacturing industries.So, we used Eqs.( 18) and ( 19) to calculate the spatial distribution of the CCT. Figure 8 shows the 2D Azimuth equidistant projection of CCT, and it was used to compare the CCT uniformity of different packages.Although  the "Hemisphere EHC" was reported as owning the best CCT uniformity 71 .Figure 8 shows "conformal w/o lens" and "cup" have better CCT uniformity.It is because the performance of uniformity highly depends on package parameters and package quality.We deliberately change the package parameter to degrade the CCT uniformity of "Bias Hemisphere" and "Hemisphere EHC".The package of "Bias Hemisphere" shows the CCT distribution of higher CCT in the right side, as shown in Fig. 8a.Since the short wavelength contributes to higher CCT, this picture is coincident with the wavelength distribution as shown in Fig. 3b-d.
Figure 9a shows the color coordinate measured along the angle φ = 0 .Where the red curves are measured by the ODSG, and the blue curves are measured by the SISH.It shows most SISH curves are coincident with the ODSG curves within the detection region |θ T | < 80 • , except some regions within 60 • < |θ T | < 80 • .The errors are large inaccuracy induced by the weak irradiance, as |θ T | gets larger.Figure 9b shows the calculated CCT, based on the color coordinates.Therefore, the mismatch between the ODSG and the SISH appears in the similar regions of Fig. 9a.Table 2 shows the AD and the DSD calculated from the CCT curves within |θ T | < 60 • .It shows the maximum value of |AD| and |DSD| are smaller than 69k.

Conclusion
The lighting and display manufacturing industries need high throughput measurement in either color accuracy or spectrum distribution.The SIS meter was proposed to make high-speed whole-field light-distribution measurement.However, it suffered from stray-light noise and lacked the capability for spectrum measurement.We proposed a Snapshot Hyperspectral technology to build up a SISH.The screen of the SISH was replaced by SSS.There are totally 2233 holes in the SSS, and was used to sample the intensity distribution sparsely.It makes the scattering reflection of the screen decreasing to 0.6% , and the SRSL noise is depressed accordingly.In the proposed snap-shot hyperspectral, the designed spectral resolution is 5nm, and the designed angular resolution is 0.9 • to 1.2 • .
In the spectrum retrieval process, the laser beams with wavelengths 457 nm, 532 nm, and 671 nm were used as light sources to get the dispersed images of SSS, i.e., G r (ξ , η) , G g (ξ , η) , and G b (ξ , η) .Then, the three images were used to calculate the binominal equations to retrieve the hyperspectrum raw data SSS 0 (x 0 , y 0 ; ) from a captured photo G(ξ , η).
In the experiment, the measured spectral resolution is 10.1 nm.We compared the hyperspectral and CCT distribution measured by SISH with which measured by ODSG.Four types of LED packages were used as samples.It  shows that for all types of packages, NCC values of the hyperspectral are higher than 95% in the detection region |θ T | < 80 • .AD and DSD calculated from the CCT curves within |θ T | < 60 • have the maximum values of 69k.

Figure 2 .
Figure 2. (a) The system setup of the snap-shot hyperspectral; (b) the holes distribute in square grid; (c) the dispersion light of the square-grid holes overlaps to each other; (d) the improved holes distribution; (e) the dispersion light of the improved holes was separated from each other.

Figure 3 .Figure 4 .
Figure 3. (a) The coordinates in the Azimuth equidistant projection and the coordinate for the one-dimension expression, and (b-d) the hyperspectal of a bias hemisphere package LED measured by the SISH along the azimuth angle φ = 0 • , 45 • , and 90 • , respectively.

Figure 5 .
Figure 5.The measurement of the spectral resolution, (a) the captured image; (b) the point spread function of wavelength; (c) the statistical result of FWHM over total 2233 points.

Figure 9 .
Figure 9.The comparison of (a) color coordinates, and (b) CCT.

Table 2 .
The AD and the DSD calculated from the CCT within |θ T | < 60 • . .